Section: New Results

Automatic Differentiation and Dynamic Memory

Participants : Valérie Pascual, Laurent Hascoet, Jean Utke [Argonne National Lab. (Illinois, USA)] .

Adjoint differentiated code obtained by source transformation (OpenAD, tapenade ...) consists of a forward sweep that essentially copies the original code, and a backward sweep that computes the derivatives, These two sweeps must have the same control flow shape, only reversed. The allocation and deallocation of some dynamic memory inside the forward sweep requires a similar pattern in the backward sweep. However, allocations do not always return the same memory chunk, and therefore all memory addresses must be updated to preserve their consistency in the backward sweep.

This problem can only be solved dynamically, at run-time. A compile-time analysis would have to be conservative, implying many overapproximations and in the end an unreasonably inefficient adjoint code. Our approach is thus to design a library that encapsulates all calls to memory allocation primitives (malloc, free ...) in order to register the allocated addresses and to be able to restore consistency of pointers during the backward sweep. This strategy is similar to the one we use for MPI calls, cf  6.1 , and is actually needed in our AMPI strategy. All we can hope from a static analysis is to detect the simple cases where addresses could be recomputed instead of stored and updated. This may apply to a significant portion of memory manipulalions, and may thus reduce the overhead due to the dynamic updating.

We started developing this library, called ADMM for Adjoinable Dynamic Memory Management. tapenade will eventually produce adjoint code that calls these primitives instead of the standard memory management primitives.